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Abstract. The aims of concept-oriented reading 
instruction (CORI) are to increase motivation, 
strategies, and conceptual learning. To attain these 
aims, CORI classrooms are: conceptually oriented, 
observational, collaborative, and personalized, 
emphasizing strategies of searching, comprehending, 
integrating, and composing for audiences of peers. 
We conducted two quasi-experimental studies com- 
paring CORI in language arts and science in grades 
3 and 5 to traditional instruction on two perfor- 
mance assessments. In Study 1, CORI students from 
three schools were higher in literacy engagement, 
conceptual learning, and conceptual transfer than 
traditional students, controlling three background 
variables. In multiple regressions, the variables of 
instruction, literacy engagement, and student back- 
ground accounted for approximately 50% of the 
variance in conceptual learning. In Study II, CORI 
students were higher than traditional students in 
measures of literacy (reading and writing com- 
bined), language use, science, and social studies, 
but not in math, which was not taught in CORI. 
These experimental findings suggest that the princi- 
ples of integrated teaching in CORI are responsible 



for increased literacy engagement and conceptual 
understanding in language arts and science. 

STUDY I 
Introduction 

It has been emphasized from a variety of 
theoretical perspectives that reading is inher- 
ently a meaning-making process (Barr, Kamil, 
Mosenthal, & Pearson, 1991). However, the 
nature of meaning constructed from text may 
vary dramatically from lower-order, rote, 
verbatim meanings to higher-order, interpre- 
tive, and conceptual meanings. As students 
read, they gain important facts and bits of 
information. At the same time, motivated 
readers construct explanations and attempt to 
account for the particular elements they have 
comprehended. In this paper, we suggest that 
when students are gaining conceptual knowl- 
edge from text they are forming higher-order 
principles that explain and account for particu- 
lar features of text they have understood. 
Forming these principles is effortful, requiring 
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a broad repertoire of cognitive strategies that 
can be flexibly applied. 

These complex constructions of meaning 
depend on motivations for understanding (Paris 
& Turner, 1994). Acknowledging the impor- 
tance of cognitive capabilities, such as using 
prior knowledge (Anderson & Pearson, 1984), 
activating cognitive strategies (Collins-Block, 
1992), and following metacognitive directives 
(Baker & Brown, 1984), we believe that in- 
cluding the role of motivational variables is 
valuable (Pintrich & Schrauben, 1992; Wig- 
field & Eccles, 1992) in accounting for expertise 
in reading and writing. Not only is motivation 
expected to influence the amount and breadth 
of students’ reading activities (Wigfield & 
Guthrie, 1995), but also motivation is likely to 
enhance the construction of higher-order 
meanings from text. 

Positive motivational dispositions toward 
using strategies and developing explanations, in 
contrast to merely finishing tasks and recording 
elemental bits of information, are necessary to 
the acquisition of explanatory understanding. 
Consequently, we agree with Bereiter and 
Scardamalia (1989) that these motivations and 
dispositions should become goals of instruc- 
tion. In this paper, we examine the relation- 
ships of strategies, motivations, and conceptual 
learning, and we ask whether instructional 
contexts influence the development of literacy 
engagement (i.e., motivated strategy-use) and 
conceptual learning (see Figure 1). 

Perspectives on Conceptual Learning 

At least three perspectives on conceptual 
learning can be identified in the current litera- 



ture. The first perspective emphasizes the 
enrichment of mental structures from a con- 
structivist perspective (Glynn, Yeaney, & 
Britton, 1991). For example, Chi, DeLeeuw, 
Chiu, and Lavancher (1994) reported that 
higher levels of conceptual knowledge about 
the biology of the human heart were associated 
with enriched mental models. These models 
contained a relatively large number of features, 
and a systemic, dynamic sense of how the heart 
operates. Conceptual learning was associated 
with an increased number of features, com- 
pleteness of the system, and intersection among 
the heart’s functions. In addition to showing 
that individual differences between learners 
could be described in terms of different levels 
of mental models, Chi et al. (1994) showed 
that asking students to explain individual facts 
and propositions as they read a text enhanced 
conceptual learning. Consistent with this view, 
Mayer (1992) stated that conceptual knowledge 
can be described as a runable mental model. 
His studies illustrated that conceptual under- 
standing of a system such as a bicycle pump 
consists of knowing the parts, the action of the 
parts, and the principles that govern those 
actions including friction, resistance, and 
motion. Students who learned these principles 
were able to answer high-level questions and 
solve problems better than students who did 
not possess such a dynamic mental model. 

The enrichment view of conceptual learning 
from text is also espoused by diSessa (1993) 
who argued that learning the principles of 
physics entails the development of a sense of 
how systems work. He stated that “understand- 
ing evolves toward compactness involving a 
few principles that are as general as possible” 
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Figure 1. Promoting conceptual learning through literacy engagement and integrated instruction. 
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(p. 190). He concluded that “cultivating the 
sense of mechanism (how systems work) can 
help students to develop adequate scientific 
models of situations” (p. 206). Understanding 
biology can also be construed as conceptual 
learning. In a review of research on knowledge 
representation, Alexander, Schallert, and Hare 
(1991) concluded that as students become more 
knowledgeable in a subject, their understanding 
is increasingly organized around fundamental 
principles. These principles are associated with 
rules of evidence and procedures for gathering 
and testing evidence. Both diSessa (1993) and 
Alexander et al. (1991) underscore the view 
that conceptual learning consists of generaliza- 
tions that embrace particular features, struc- 
tures, and functions. 

Researchers who examine children’s concep- 
tual development concur with this perspective. 
For example, Keil (1989) documents that a 
characteristic-to-defining shift describes the 
development of concepts of everyday life. 
Young children shift from evenly weighing 
several features of an object such as an island, 
to placing heavy emphasis on one or two 
primary features and fundamental organizing 
dimensions. For instance, an island has trees, 
land, animals, and water; but the defining 
feature of an island is that it is surrounded by 
water. This shift is confirmed by Metz (1995) 
who argues that biological concepts depend on 
classification based on defining features, but 
higher-order principles such as photosynthesis 
rely on principled organization of structures, 
functions, relations, and their change across 
time. As Reif (1990) says, the concept is a set 
of principles or rules that enable the person to 
understand specific properties, the values of 



those properties, and how factors influence 
them and their interrelationships. A variety of 
techniques now exist to represent those en- 
riched knowledge structures (Jonassen, Beiss- 
ner, and Yacci, 1993). 

A second perspective on conceptual learning 
emphasizes restructuring of existing information. 
Vosniadou (1994) presents a vivid illustration 
of the restructuring process. She documents 
that across grades 1-6, children restructure 
their knowledge of the earth’s shape. Most 
younger students believe a dual earth theory in 
which the earth is round but you can fall off of 
it if you walk far enough. By age 8, one-third 
of the students believe the earth is a sphere; by 
age 10, two-thirds believe the earth is a sphere 
that peoples on all sides can occupy. Concep- 
tual learning, in this perspective, entails the 
revision of one form of explanatory under- 
standing into a different form of understanding. 
The particular facts and features of the concept 
change little, but the principle of explanation 
changes substantially. 

Strike and Posner (1985) describe the pro- 
cess of restructuring. They state that the pro- 
cess consists of “dissatisfaction within an 
existing conception, followed by finding a new 
conception understandable, leading to an initial 
belief in its plausibility, and concluding with 
the belief that the new conception is ultimately 
fruitful” (p. 221). Extensive research on per- 
spective was reported in a review by Chinn and 
Brewer (1993). As Smith (1991) noted, the 
problem of conceptual change is that “students 
have to change an idea they had initially” (p. 
49). The restructuring perspective underscores 
the need to challenge old knowledge and ac- 
commodate it to fit new data. 
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The third perspective defines conceptual 
learning as the cumulative increase of informa- 
tion. In studies of knowledge acquisition from 
text (Carroll & Freedle, 1972), free-recall tasks 
are used to measure students’ recollection of 
text-based propositions, irrespective of the 
higher-order or lower-order nature of that 
meaning. In these studies, conceptual learning 
was represented on a continuum from low to 
high information levels. In science education, 
too, this cumulative perspective is found. For 
example, Novak and Musonda (1991) reported 
that concept maps (graphic organizers) can 
document learning by showing an increase in 
the number of nodes and the number of links 
between the nodes. These increases are char- 
acterized as conceptual, or meaningful, learn- 
ing. Spoehr (1994) reported that the knowledge 
of science concepts can be represented as a 
concept map of nodes and links and that the 
number of these elements can be shown to 
increase with appropriate instruction. Con- 
firming this perspective, Krajcik (1991) argued 
that learning concepts of chemistry can be 
represented in the form of increasingly elabo- 
rated concept maps. 

The distinction between the cumulative 
perspective and the enrichment view is that the 
cumulative orientation does not underscore 
higher-order principles. In the cumulative 
perspective, the nodes and links are given equal 
weight. Qualitative shifts in types of knowledge 
are not recognized. The cumulative perspective 
differs from the restructuring perspective by 
emphasizing the accretion of information rather 
than revision of existing knowledge formations . 

In this report, we adopt the enrichment 
perspective in which knowledge increases in 



several respects. As students enhance their 
conceptual understanding, they gain command 
of more elements, features, and functions in a 
particular domain. Learners form principles to 
organize these particulars into relationships. At 
the highest levels of conceptual understanding, 
students possess interrelated, explanatory 
principles for phenomena and events. They 
gain the ability to coordinate theory and evi- 
dence, and they can express the interplay 
between them. See Figure 1 for a schematic of 
the variables in this study. 

Relations of Strategies and Motivations to 
Conceptual Learning 

Learning concepts from text is strongly 
facilitated by complex cognitive strategies. As 
many investigators have shown, gaining knowl- 
edge from text depends on problem finding 
(Collins-Block, 1992), applying prior knowl- 
edge (Anderson & Pearson, 1984), searching 
for information (Armbruster & Armstrong, 
1993), comprehension strategies (Dole, Duffy, 
Roehler, & Pearson, 1991), self-monitoring 
(Baker & Brown, 1984), and strategies suitable 
for different genre (Graesser, Golding, & 
Long, 1991). 

Although these cognitive strategies are 
helpful in learning concepts from text, such 
strategies depend heavily on intrinsic motiva- 
tion. In a review of motivation literature, 
Pintrich and Schrauben (1992) documented the 
high correlations between intrinsic motivations 
and strategy use. Including interest in the 
content, wanting to learn for its own sake, and 
feeling immersed in the literacy task (Reed & 
Schallert, 1993), various intrinsic motivations 
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are associated with more frequent and appro- 
priate use of higher-order cognitive strategies. 
These findings were confirmed experimentally 
by Graham and Golan (1991) who showed that 
more meaningful tasks produced higher intrin- 
sic motivations and the use of more complex 
learning strategies than less meaningful tasks. 
Unfortunately, none of these investigations 
examined whether the more motivated students 
who were using relatively higher numbers of 
strategies were, in fact, gaining higher levels of 
conceptual knowledge. We do not know from 
these studies whether conceptual learning was 
enhanced by motivation. This literature does 
confirm, however, that strategies that are 
traditionally associated with conceptual learn- 
ing are strongly associated with intrinsic moti- 
vation. 

One motivational variable (text-based, 
situational interest) has been related to concep- 
tual learning. As defined by Schraw, Bruning, 
and Svoboda (1995), situational interest is 
“common across individuals, short-lived and 
elicited within a particular context” (p. 1). 
Situational interest contrasts with personal 
interest that is “unique to the individual, topic 
specific, long-lasting, and exits before encoun- 
tering a particular text” (p. 1). Hidi and Ander- 
son (1992), and Shraw et al. (1995) have 
documented that students who display high 
situational interest in a particular text will show 
relatively elaborated understanding of concepts 
within the text. Unfortunately, neither of these 
studies controlled for amount of prior knowledge 
about the topic, which is arguably associated 
with situational interest. However, Schiefele 
(1992) succeeded in controlling the amount of 
prior knowledge and documented the effect 



of situational interest on conceptual knowledge 
gain from text. Although these studies docu- 
ment that situational interest is a factor in 
conceptual learning, they do not relate personal 
interest and more permanent intrinsic motiva- 
tions to conceptual learning. There is a need, 
therefore, to relate conceptual learning, strat- 
egy use, and intrinsic motivations to each other. 

Effects of Educational Context on Literacy 
Engagement and Conceptual Learning 

In a valuable review of integrated instruc- 
tion, Lipson, Valencia, Wixson, and Peters 
(1993) reported that integrated curricula are 
usually claimed to make learning meaningful, 
and this meaningfulness is thought to increase 
conceptual knowledge. Integrations of different 
disciplines (e.g., reading with writing, or 
language-arts with social studies) are claimed, 
furthermore, to accelerate the development of 
metacognitive awareness and to foster positive 
attitudes toward literacy. Similar claims are 
made by other investigators who have designed 
and implemented integrated instruction in 
science and language arts (Blumenfeld, 1992; 
Brown, 1992). However, relatively few inves- 
tigators have attempted to collect information 
regarding children’s learning and development 
in integrated instructional environments. Whe- 
ther or not students learn is a significant issue, 
according to Brophy and Alleman (1991) who 
caution that integrated instruction may not do 
justice to the separate disciplines combined. 
We concur that trivial forms of thematic teach- 
ing may undermine rather than embellish the 
learning of conceptual themes and fundamental 
literacy processes. 
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Characteristics of the classroom context and 
motivation for literacy have been reported by 
Turner (1995) in a study of first graders and 
Hooper (1994) in an investigation of third, 
fourth, and sixth graders. After observing 
primary classrooms intensively, Turner (1995) 
reported that motivation for literacy was affect- 
ed by two types of classroom tasks. Open 
tasks, in which students experienced relatively 
high levels of choice, challenge, control over 
learning, and collaboration with other students, 
were associated with high reports of intrinsic 
motivation. In relatively closed tasks, the 
teacher determined the learning activities (low 
choice), tasks were simplified and rote (low 
challenge), the program defined the tasks 
(low control), and students worked individually 
(low collaboration). In closed tasks, students 
reported less intrinsic motivation, less effort, 
and lower persistence. 

Confirming the importance of challenge, 
Hooper (1994) reported that when literacy 
tasks contained an expectation for higher-order 
activity, such as analyzing books, students 
were more likely to be intrinsically motivated 
than when tasks were simpler, at the word and 
sentence level. Intrinsic motivation, which 
referred to involvement and enjoyment, was 
apparent on 72% of complex tasks; whereas 
extrinsic motivation, which referred orientation 
to task completion, occurred for 72% of the 
simple tasks. These findings generalized across 
grades 3-6 and included learning disabled as 
well as mainstreamed students. These two 
investigations reported a prominent association 
between task structure and intrinsic motivation 
in extant classroom contexts. 



In addition to the suggestion that challeng- 
ing, personalized, collaborative classrooms 
enhance motivation for learning, there is a 
growing consensus that these contexts enhance 
conceptual understanding. For example, Blum- 
enfeld (1992) and Perkins (1992) contend that 
integrated curricula can enable students to gain 
deep understanding of content. Unfortunately, 
there is very little evidence to support this 
claim. Blumenfeld and Meece (1988) and 
Brown (1992) show that when conceptual 
themes are emphasized, rather than activities, 
tasks, or procedures, students gain higher- 
order knowledge relatively well. We concur 
and extend the proposition to literacy engage- 
ment and conceptual learning. We propose that 
enhancing literacy engagement through con- 
cept-oriented reading instruction (CORI) can 
enhance students’ literacy engagement which 
will increase their ability to learn high-level 
concepts. To examine this proposition, we 
conducted two studies in which we compared 
CORI students to control groups. The follow- 
ing questions guided the study. 

1 . Does literacy engagement increase concep- 
tual learning when “background” factors of 
science background, literacy level, and topic 
knowledge are controlled? 

2. Does CORI increase literacy engagement 
when background factors are controlled? 

3. Does CORI increase conceptual learning 
when literacy engagement and background 
factors are controlled? 

4. How do students in grades 3 and 5, who 
receive CORI, compare to other students in 
literacy engagement? 

5. How do students in grades 3 and 5, who 
receive CORI, compare to other students in 
conceptual learning? 
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6. How does conceptual transfer relate to con- 
ceptual learning and literacy engagement in 
grade 5 and grade 3 when background 
factors are controlled? 

7. What effect does CORI have on compre- 
hension of informational and narrative text? 

8. What instructional principles distinguish 
CORI and traditional instruction? 

Method 

Participants 

Teachers and schools. Three schools on the 
border of a large metropolis in the mid- Atlantic 
states participated in this study. Each school 
had multicultural populations consisting of 
approximately 50% African-American, 25% 
Caucasian, 15% Hispanic, and 10% Asian. 
Two of the schools were designated as Chapter 
1 and one school had a mainstream program 
for orthopedically disabled students. The 
schools were nominated by the district supervi- 
sor of reading as likely to benefit from an 
integrated curriculum for low achieving stu- 
dents, and the principals were pleased to par- 
ticipate after being asked by the project direc- 
tor. The teachers were volunteers from the 
schools, and were approved by the principal 
and the project director. They were not nomi- 
nated as exemplary instructors, but were selected 
as typical, experienced elementary teachers. 

Classrooms and students . Third-grade class- 
rooms in all schools were self-contained, with 
approximately 30 students in each. Due to 
the transitory population, a total of approxi- 
mately 20 fifth-grade students in each of two 
classrooms completed the year of CORI. A 



total of 20 fifth-grade students from each of 
two basal classrooms was used as comparison 
students. Approximately 25 third-grade stu- 
dents in each of three classrooms completed the 
year of CORI; approximately 75 students in 
third-grade basal classrooms were used for 
comparison. The students consisted of approxi- 
mately equal number of boys and girls with 
5-6 special education students mainstreamed 
into each classroom. Learning disabled, ortho- 
pedically disabled, and emotionally disturbed 
learners were included in the instruction and 
assessments. Low achievers from transitory 
home backgrounds were also included. 

Design 

This quasi-experiment consisted of two 
instructional conditions , CORI and traditionally 
organized basal and science instruction. CORI 
was implemented in five classrooms, 3 third- 
grade and 2 fifth-grade classrooms. Comparison 
classrooms within each school were selected 
for comparison to the CORI classrooms. These 
classrooms were selected on the basis of com- 
parable students, teachers, and school settings. 
The measures of literacy and engagement 
including the performance assessment, ques- 
tionnaires, and teachers’ ratings were adminis- 
tered to all 10 classrooms. The design was a 
post-measure only in which the performance 
assessment and questionnaires were adminis- 
tered in the first half of April to all students. 
Teachers’ observational ratings of students’ 
levels of English/language arts and science 
backgrounds were obtained in the fall of 1994 
as statistical controls. 
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Instructional goals . The CORI and tradition- 
al programs were both directed toward similar 
goals for English/language arts and science. 
These goals were approved by the district, 
school, and project participants. In Eng- 
lish/language arts, the objectives consisted of 
interpreting stories, comprehending expository 
texts, locating and integrating information from 
multiple texts, summarizing, self-monitoring, 
writing personal narratives, composing infor- 
mational reports, and writing poetry. In 
science, the common goals included under- 
standing the life cycles of plants and animals, 
describing important adaptations of animal spe- 
cies, understanding cycles of weather and 
seasons, collecting data, interpreting graphs 
and tables, and interpreting data from class- 
wide projects. More complex higher-order 
goals were adopted for fifth-grade than for 
third-grade students; and instruction was ad- 
justed to meet the needs of all learners, includ- 
ing approximately five students in each grade 
3 CORI classroom who entered the year read- 
ing at a 1.5 grade level. These goals and the 
instruction are presented more fully in Guthrie, 
McCann, Hynd, and Stahl (in press). 

Concept-oriented reading instruction. The 
framework for CORI included four phases: 
observe and personalize, search and retrieve, 
comprehend and integrate, communicate to 
others. To implement this framework, teachers 
first identified a conceptual theme for instruc- 
tional units to be taught for 16-18 weeks in the 
fall and spring. The themes selected by third- 
grade teachers consisted of the adaptations and 
habitats of birds and insects. The third-grade 
units in the spring consisted of weather, sea- 
sons, and earth formations. Fifth-grade units in 



the fall related to life cycles of plants and 
animals, and the spring units emphasized earth 
science, including the solar system and geolog- 
ical cycles. 

At the beginning of each unit students 
engaged in observation and hands-on activities 
both outside and inside the classroom. Third- 
and fifth-graders participated in such activities 
as collecting and observing crickets, construct- 
ing spider webs, dissecting owl pellets, and 
building weather stations . Within each activity , 
students personalized their learning by compos- 
ing their own questions as the basis for observ- 
ing, reading, and writing. Student questions 
included a structural focus, such as, “How 
many types of feathers does a bird have?” 
Then conceptual questions, such as “Why does 
that bird have such a long beak?” evolved as 
students attempted to explain the phenomena 
they had observed. These questions generated 
opportunities for self-directed learning. Stu- 
dents chose their own subtopics, found particu- 
lar books, selected peers for interest-based 
activities, and constructed their goals for 
communicating to others. 

The second phase of the CORI framework 
consisted of searching and retrieving informa- 
tion related to the student’s questions. Students 
were taught how to use the library, find infor- 
mation books, locate information within expos- 
itory texts, and utilize a diversity of community 
resources. In addition, direct strategy in- 
struction was provided to help students inte- 
grate across information sources including 
texts, illustrations, references, and human 
experts. In addition to informational texts, 
literature including stories, folklore, novels, 
and poetry were woven through the instruction. 
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Most of the teachers began the units with a 
narrative related to the theme that students read 
at the same time they were conducting science 
observations . Following observing and forming 
conceptual questions, teachers moved to infor- 
mational texts. As students concluded their in- 
depth study of multiple informational texts, 
teachers introduced novels, chapter books, and 
poetry related to the conceptual theme of the 
unit. 

The last phase of the CORI framework is 
communicating to others. Having gained exper- 
tise in a particular topic, students were moti- 
vated to speak, write, discuss, and display their 
understanding to other students and adults. In 
both third- and fifth-grade classrooms, students 
made posters, wrote classroom books, and 
composed extended displays of their knowl- 
edge. One class made a videotape of its weather 
unit, providing a lesson on weather prediction 
and explanation for the rest of the school. A 
more thorough description of the CORI 
instructional framework is provided elsewhere 
(Guthrie, McGough, Bennett, & Rice, 1996). 

The teachers in the traditional classrooms 
followed their usual pattern of using the teach- 
ers’ guide and the sequence of content and 
activities in the basal program. Students an- 
swered the end of unit questions, and were 
provided materials that matched their reading 
levels. Science content in the third- and fifth- 
grade basal-reading classrooms was directed to 
similar objectives as the CORI classrooms. 
Topics of adaptation, life cycles, weather and 
seasons, and solar systems were taught. The 
basal teachers in all of the schools were fre- 
quent visitors to the CORI classrooms and 
adopted some texts and teaching approaches 



used in the CORI classrooms. This sharing 
may have lead to an underestimate of the 
distinctiveness of the CORI program, and to a 
conservative quasi-experimental comparison. 

Performance Assessment of 
Reading/Language Arts 

The performance assessment was designed 
to measure seven aspects of literacy engage- 
ment. More information is available in Guthrie 
et al. (1996). The assessment was administered 
during a 1-week period in each basal and CORI 
classroom by the teachers. All students took 
the assessment in the same 3-week period at 
the beginning of April 1995. Students worked 
approximately 1 hr daily for 5 days. The 
assessment consisted of a learning context with 
seven tasks. The tasks will be described here, 
and rubrics for coding performance are pre- 
sented in the Appendix. The assessment was 
introduced with an observational activity. 
Students were given a colored picture of a 
pond with fish, insects, birds, and trees. With 
a partner, students discussed all the things they 
observed in the picture for 5 min. 

Prior knowledge . After viewing the picture, 
students worked independently to write their 
knowledge of the topic before' entering the 
search phase. Students were asked to “explain 
how ponds are different from deserts” and 
were aided in the task by being asked, “What 
is a pond like? What is a desert like? How are 
they different?” All students were given suffi- 
cient time to finish their writing. 

Search. Students were next given an oppor- 
tunity to search for ideas and information about 
ponds and deserts. Their task was to “explain 
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the differences between ponds and deserts.” 
Students were given a 46-page booklet contain- 
ing 14 sections with 1-5 pages of information. 
A table of contents, index, and glossary were 
provided. Four of the sections were not directly 
relevant to the question and 10 were directly 
useful. Of the 10 relevant sections, 5 were 
appropriate for third-grade students, and 5 
were more likely to be appropriate for fifth- 
grade students. 

Students were given a log to fill out during 
the search activity. In the log, they were asked 
to present which section they selected, their 
reasons for choosing this information, and their 
notes on what they learned from the section. 
Students were given an unlimited amount of 
space to fill out their log. The search took 
place during a 1-hour period on 2 different 
days, allowing students as much time as they 
thought they needed to complete the activity. 
Students who finished early were permitted to 
rest or read a book of their choice. 

Drawing. Students were first asked to 
“Draw a picture to show how ponds are differ- 
ent from deserts. ” Students were asked to label 
the important parts and given ample time for 
the activity. 

Writing. After completing the drawing, 
students were given a writing task with the 
following instructions, “Write an explanation 
of how ponds are different from deserts.” 
Asked to write about the important parts of 
ponds and deserts and use science ideas in their 
explanations, students were given unlimited 
time for this activity. Following the writing 
task, students were given six specific questions 
to probe their knowledge of these biomes more 
deeply. The questions addressed particular 



animals, their habitats, and their adaptations to 
the habitat. 

Conceptual transfer. Students were asked to 
perform a task that required them to transfer 
their knowledge from the previous learning 
activities. The instructions were as follows, 
“Suppose some people drink all of the water 
from a pond one day, would the pond now be 
like a desert? Please explain your answer.” 
Students were not permitted to refer to their 
previous drawings, writings, or texts. They 
were given ample time to complete the activity 
and encouraged to think extensively as they 
performed the activity. 

Informational text comprehension. Students 
were given an expository text containing prose, 
a diagram, and an illustration. They were 
given instructions to read this material and 
answer four questions. The first question 
required the students to integrate information 
across all three forms of information in the 
task. The next three questions asked students to 
address the illustration, the text, and the dia- 
gram in that order. Space was given for each 
answer separately. 

Narrative interpretation. In this stage, 
students were provided a narrative of approxi- 
mately 1,000 words. An age-appropriate, 
authentic short story was given for each grade 
level. Students answered three questions, the 
first addressing recall of a particular event, the 
second addressing the author(s) perspective on 
one character, and the third addressing the 
theme of the literary selection. 

Performance Assessment Coding Rubrics 

Coding rubrics were constructed to classify 
the students’ responses on each task of the 
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performance assessment. To build the rubrics, 
we started with the writing task. We sorted the 
responses of 25 students holistically into six 
relatively higher and lower categories. We then 
identified the critical qualities of each level. An 
independent graduate student rated 25 re- 
sponses according to these categories to examine 
the interrater agreement. Having established a 
rubric for the writing task, the rubric was used 
for prior knowledge, drawing, and transfer. 
The search rubric was constructed in a similar 
manner. Twenty-five responses were sorted 
into six categories to measure reliability. The 
interrater agreements for adjacent and exact 
coding into these categories, respectively, 
were: prior knowledge (96%, 84%), search 
(92%, 84%), drawing (96%, 88%), writing 
(96%, 80%), conceptual transfer (92%, 80%), 
informational text comprehension (100%, 
85%), and narrative interpretation (95%, 
90% — third grade; 100%, 85% — fifth grade). 

Motivational and Strategy Questionnaire 

Following the search activity, students were 
given a 35-item questionnaire about their 
motivations and strategies for performing the 
search activity. One set of questions addressed 
intrinsic motivations for the task (e.g., “After 
reading one packet about ponds and deserts, I 
wanted to read more”). The second set of 
questions elicited the students’ use of strategies 
(e.g. , “After I took some notes, I went back to 
the packet to make sure I didn’t miss any- 
thing”). The response mode was: (1) not like 
me at all, (2) a little like me, (3) sort of like 
me, (4) a lot like me. Students were instructed 
that they should think about how they felt while 
they were searching and answer accordingly. 



Teacher Observational Rating 

To construct a measure of students’ general 
level of literacy and science background at the 
beginning of the year, teachers conducted 
observational ratings. For each child in the 
basal and CORI classrooms, the teacher rated 
each child according to her quintiles in literacy 
and science, using the grade-level students in 
the school as the reference group. These rat- 
ings were used as a covariate in some of these 
statistical analyses to adjust the traditional and 
CORI classrooms. 

Instructional Questionnaire and 
Intervention Check 

To determine the extent to which the princi- 
ples of CORI were being implemented, we 
developed an instructional questionnaire. Based 
on analyses of videotaped sessions of the CORI 
and focal groups with teachers in the previous 
year, we identified seven dimensions of the 
CORI learning context including: (a) observa- 
tional experiences related to science phenom- 
ena, (b) an orientation towards conceptual 
knowledge, (c) student-direction, (d) a collabo- 
rative atmosphere among the learners, (e) 
explicit teaching of cognitive strategies to 
improve reading and writing, (f) opportunities 
for different modes of personal expression of 
knowledge, and (g) the integration of science 
topics with language arts processes. 

The research team wrote 99 items which ad- 
dressed each of the seven dimensions. The 
response format was a likert scale ranging from 
1 to 4. Sometimes the response choices were 
worded as “very true of my class” to “not at 
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Table 1 

Teacher Questionnaire on COR1 and Traditional Teaching 



Instructional Dimension 


No. of Items 


Reliability 


Correlation with 

Instructional 

Program 


Real-World Experience 
(Observational vs. Abstract) 


9 


.90 


.97 


Knowledge 

(Conceptual vs. Factual) 


12 


.89 


.97 


Direction of Control 
(Student vs. Teacher) 


15 


.91 


.95 


Social 

(Collaborative vs. Individual) 


13 


.66 


.86 


Strategies Taught 
(Explicitly vs. Implicitly) 


9 


.75 


.61 


Written Expression 
(Personal vs. Conventional) 


9 


.47 


.20 


Integration of Language Arts & 
Science 

(Integrated vs. Departmentalized) 


8 


.71 


.90 



m p < .001 

all true of my class.” Other items asked the 
teachers to estimate the time invested in an 
instructional practice with choices ranging from 
“almost every day” to “once a month or less.” 
A trial version of the questionnaire was given 
to the 5 CORI and 5 comparison teachers for 
review. They were asked to identify items that 
they felt reflected their own teaching practices 
most effectively. The participants were also 
invited to give written feedback about items on 
the questionnaire. As a result of the teachers’ 
comments, some items were changed, reor- 



dered, or reverse-coded. The resultant ques- 
tionnaire was administered to the participants 
in the spring of the academic year. Fourteen 
items were excluded from analyses because of 
missing data. Six additional items were with- 
held because they showed low variability. 

This instructional questionnaire contained 
seven bipolar scales. For example, the scale on 
direction of instructional control ranged from 
student-directed (more CORI-like) or teacher/ 
program-directed (more traditional). Items in 
this scale were coded to reflect either an 
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emphasis on student direction or an emphasis 
on teacher direction. Questionnaire data are 
displayed in Table 1 . Reliabilities are generally 
adequate, except written expression. Correla- 
tions with instructional type were all signifi- 
cant, except written expression. A positive 
correlation indicated that CORI teachers em- 
phasized the left side of the bipolar dimension; 
whereas traditional teachers emphasized the 
right side of the dimension. The CORI versus 
traditional emphasis is shown parenthetically in 
Table 1. 

Procedures 

Students in the CORI classrooms received a 
full academic year of this teaching, and the 
control students received traditional instruction 
throughout the year. Teachers had full respon- 
sibility for all instruction, and participated in 
an inquiry workshop monthly, led by the 
University researcher, to share progress and 
problems. The performance assessment was 
given by the teachers in the first half of April. 
Traditional students performed comfortably on 
the performance assessment, finding it to be an 
enjoyable activity. Coding of the performance 
assessment was completed by the graduate 
students and the first author. 

Results 

The data were analyzed to explore the model 
in Figure 1 and to address each question at the 
end of the introduction. Measures from the 
performance assessment and teachers’ ratings 
were used as indicators of the constructs in 
Figure 1. Conceptual learning was measured 



with the combined writing and drawing score 
because students often referred to their draw- 
ings in their written work. The indicator of 
literacy engagement was the score on the 
search task, which consisted of extended read- 
ing and notetaking from multiple, illustrated 
texts. On the search task, strategies of search- 
ing, comprehending, integrating, and summa- 
rizing were needed. Motivational dispositions 
of interest, effort, persistence, and self-efficacy 
were expected to be beneficial in this task as 
suggested in previous studies (Guthrie, et al., 
1996). 

To measure conceptual learning with maxi- 
mally sensitive procedures, we asked students 
follow-up questions after their writing and 
drawing activity. We constructed a prompted 
score by placing students on the writing rubric 
after accounting for the follow-up questions. 
Analysis of the prompted scores showed that 
students attained significantly higher rubric 
levels on prompted than on writing-only tasks, 
t(221) = 7.94,/? <.0001. To examine whether 
the advantage varied by grade or instructional 
program, we conducted a 2 (grades) X 2 (types 
of instruction) analysis of variance (ANOVA) 
on the difference of writing and prompted 
scores. No significant effects were observed. 
Therefore, we used the writing scores in subse- 
quent analyses. 

Students’ literacy levels and general science 
knowledge in the fall of 1994 were rated by the 
teachers and reading specialists. These two 
ratings were combined to obtain a general 
literacy-science background measure. Prior 
knowledge on the topic of the assessment was 
drawn from the prior knowledge task in the 
performance assessment. 
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Question 1 

Does literacy engagement increase concep- 
tual learning when science background, literacy 
level, and topic knowledge are controlled? This 
question addresses the strength of link A in 
Figure 1 . We used the data for both grades and 
both instructional programs. To analyze the 
data, a hierarchial multiple regression was 
conducted in which the dependent variable was 
the combined score of writing and drawing 
representing conceptual learning. Independent 
variables were entered in the order of a com- 
bined background score of literacy level and 
science knowledge, prior knowledge of the 
topic in the performance assessment task, and 
the search score, which was the indicator of 
literacy engagement. 

Multiple regression analysis showed that 
search (literacy engagement) significantly 
predicted writing and drawing (conceptual 
learning) when the background variables were 
taken into account. The multiple regression was 
R = .62, ip < .0001); search (literacy engage- 
ment) accounted for 10% of the unique variance 
in writing and drawing (conceptual learning), 
which was significant at p < .0001, as 
indicated in Table 2. 

We also addressed Question 1 for the grade- 
5 and grade-3 students separately. A hierarchial 
multiple regression for combined writing and 
drawing as the dependent variable, with liter- 
acy and science background, prior knowledge, 
search, and instruction as independent vari- 
ables. In grade 5, search contributed 18% of 
the unique variance (p < .0002) in writing and 
drawing and had a multiple regresion of .63. 
For grade 3, the multiple regression was .60, 



and search contributed 4% of the unique vari- 
ance {p < .05) in writing and drawing. Refer- 
ring to Figure 1, these data showed that link A 
was significant, when links 1 and 2 were con- 
trolled. The relationship held for grade 5 and 
grade 3 separately, as well as for the total 
group. 

Question 2 

Does CORI increase literacy engagement 
when background factors are controlled? This 
question refers to link B of Figure 1 . Having 
shown that our indicator of literacy engage- 
ment (search) predicted conceptual learning 
(writing and drawing), we attempted to deter- 
mine whether CORI increased literacy engage- 
ment. To examine this question, we conducted 
a hierarchial multiple regression analysis in 
which the total search score was the dependent 
variable and the independent variables were the 
combined background literacy-science score, 
the prior knowledge score,. and the integrated 
instruction score. Instruction was entered as a 
contrast between the CORI and the traditional 
programs. A multiple regression of .50 was 
observed. Instruction accounted for 17 % of the 
variance in search after background factors 
were controlled, which was highly significant 
(p < .0001). As Table 3 shows, these findings 
were also observed for grades 5 and 3 sepa- 
rately. For grade 5, instruction accounted for 
20% of the unique variance (p < .0002); and 
for grade 3, instruction accounted for 17% of 
the unique variance (p < .0002). These find- 
ings show that CORI enhanced literacy engage- 
ment, as indicated by search, in both grade 5 
and grade 3. Referring to Figure 1, link B was 
confirmed to be important by this analysis. 
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Table 2 

Multiple Regression Analyses for Questions 1 & 3 



Group 


Dependent 


Independent 


R 


R 1 


R 2 Cha 


SigCha 


Total 


Write & Draw 


Literacy/Science 


.35 


.12 


.12 


.0001 


Group 




Prior Knowledge 


.53 


.28 


.15 


.0001 






Search 


.62 


.38 


.10 


.0001 






Instruction 


.67 


.45 


.07 


.0003 


Grade 5 


Write & Draw 


Literacy/Science 


.27 


.07 


.07 


.04 






Prior Knowledge 


.47 


.22 


.14 


.002 






Search 


.63 


.40 


.18 


.0002 






Instruction 


.72 


.52 


.12 


.0007 


Grade 3 


Write & Draw 


Literacy /Science 


.39 


.15 


.15 


.001 






Prior Knowledge 


.56 


.32 


.16 


.0003 






Search 


.60 


.36 


.04 


.05 






Instruction 


.64 


.41 


.05 


.03 


Total 


Write & Draw 


Literacy /Science 


.34 


.12 


.12 


.0001 


Group 




Prior Knowledge 


.52 


.27 


.15 


.0001 






Search Total 


.60 


.36 


.09 


.0001 


Total I 


Write & Draw 


Literacy /Science 


.34 


.12 


.12 


.0001 






Prior Knowledge 


.52 


.27 


.15 


.0001 






Search— Packets 


.60 


.36 


.09 


.0001 


Total I 


Write & Draw 


Literacy /Science 


.34 


.12 


.12 


.0001 






Prior Knowledge 


.52 


.27 


.15 


.0001 






Search— Reasons 


.56 


.31 


.04 


.007 


Total I 


Write & Draw 


Literacy/Science 


.34 


.12 


.12 


.0001 






Prior Knowledge 


.52 


.27 


.15 


.0001 






Search— Notes 


.60 


.36 


.09 


.0001 



have suggested that literacy engagement (indi 
cated by the score on search) increased concep- 
tual learning (indicated by combined writing 
and drawing) and that integrated instruction 
(CORI) increased literacy engagement in com 



Question 3 

Does CORI increase conceptual learning 
when literacy engagement and background 
factors are controlled? The previous analyses 
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Table 3 

Multiple Regression Analyses for Question 2 



Group 


Dependent 


Independent 


R 


R 2 


R z Cha 


SigCha 


Total 


Search Total 


Literacy/Science 


.26 


.07 


.07 


.0003 


Group 




Prior Knowledge 


.29 


.09 


.02 


.11 




Instruction 


.50 


.25 


.17 


.0001 


Grade 5 


Search Total 


Literacy/Science 


.22 


.05 


.05 


.08 






Prior Knowledge 


.24 


.06 


.01 


.49 






Instruction 


.50 


.25 


.20 


.0002 


Grade 3 


Search Total 


Literacy/Science 


.27 


.07 


.07 


.03 






Prior Knowledge 


.34 


.12 


.04 


.08 






Instruction 


.53 


.29 


.17 


.0002 



parison to traditional teaching. We next asked 
whether instruction enhanced conceptual learn- 
ing beyond its influence on literacy engage- 
ment. To examine this question, we conducted 
a hierarchial multiple regression analysis with 
the dependent variable of combined writing and 
drawing. The independent variables were 
literacy and science background, prior knowl- 
edge of the topic, search, and instruction. To 
perform this analysis, we added instruction as 
a final independent variable to the regression 
analysis conducted for question 1 . The results 
are shown in Table 2. 

The findings for instruction as the indepen- 
dent variable were that a multiple regression of 
.67 was obtained for the total group. Instruc- 
tion contributed 7 % of the unique variance in 
conceptual learning, which was significant (p 
< .0003). When the same hierarchial multiple 
regression was conducted for grade 5, a multi- 
ple regression of .72 was obtained, and instruc- 
tion accounted for 12% of the unique variance 
( p < .0007). For grade 3, the same regression 



analysis produced a multiple regression of .64, 
with instruction accounting for 5% of the 
unique variance that was significant (p < .03). 
With reference to Figure 1, these results con- 
firm the importance of link C. The results can 
be interpreted as showing that CORI increased 
conceptual learning, in comparison to tradi- 
tional instruction. The findings suggest that 
CORI had a direct effect on conceptual learn- 
ing (link “C”) as well as indirect effect through 
literacy engagement (links B and A). 

Question 4 

How do students in grades 3 and 5, who 
receive CORI, compare to other students in 
level of literacy engagement? This question 
was addressed with a 2 (grade levels) X 2 
(instructional types) analysis of covariance 
(ANCOVA). Covariates were literacy and 
science background and prior knowledge, and 
the dependent variable was literacy engagement 
as indicated by the search score. A significant 
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effect occurred for instructional type with 
CORI students higher than students in tradi- 
tional instruction. A significant effect occurred 
for the grade, and the interaction effect was not 
significant. The findings are plotted in Figure 2. 

The ANCOVA showed that CORI was 
higher than traditional instruction in literacy 
engagement at grade 3 and grade 5. To com- 
pare the four individual means of the two 
instructional groups at the two grade levels, we 
conducted post hoc contrasts. The procedure 
consisted of coding the four groups into four 
levels of one dummy variable. One-way 
ANCOVA with Newman-Keuls contrasts were 
conducted on this variable. The result was that 
CORI grade-5 was significantly higher than all 
other groups including traditional grade 5 (p < 
.05). Next, CORI grade-3 was higher than 
traditional grade 3 (p < .05). In addition, 
CORI grade-3 was significantly higher than 
traditional grade 5 (p < .05). Finally, tradi- 
tional grade 5 was not significantly higher than 
traditional grade 3. It seems noteworthy that 
CORI grade-3 exceeded traditional grade 5 in 
this search-based measure of literacy engage- 
ment, and that students in grade 3 and grade 5 
traditional instruction were not different. 

Question 5 

How do students in grades 3 and 5 who 
receive CORI compare to traditional students in 
conceptual learning? This question was ad- 
dressed with a 2 (grade levels) X 2 (instruc- 
tional types) ANCOVA. Covariates were literacy 
and science background and prior knowledge, 
and the dependent variable was conceptual 



learning as indicated by the combined writing 
and drawing. A significant effect occurred for 
instructional type, F (1,119) = 32.61, p < 
.0001, showing that CORI students were 
higher than traditional instruction students. A 
significant effect was found for grade, F (1,1 19) 
= 5.39, p < .02. The interaction of instruc- 
tion and grade was not significant. The find- 
ings are plotted in Figure 3. 

To examine the means of the two instruc- 
tional groups at two grade levels, a one-way 
ANOVA on the four groups was conducted 
with Newman-Keuls post hoc contrasts. These 
comparisons showed that CORI grade-5 was 
significantly (p < .05) higher than all of the 
other groups, CORI grade-3 was significantly 
higher than traditional instruction grade 3 (p < 
.05). In this analysis, CORI grade-3 was not 
significantly different from traditional grade 5, 
and the two traditional groups at grade 3 and 
grade 5 were not significantly different from 
each other. 

As Figure 3 shows, CORI grade-3 was 
higher than traditional grade 3 . This difference 
was statistically significant using the Bonfer- 
roni-t statistic, assuming two comparisons 
( P < .025). Although the Newman-Keuls 
contrast did not show a significant difference, 
for this comparison, due to the exploratory 
nature of this study, we believe results of the 
Bonferroni-t statistic should be considered 
seriously. 

Question 6 

How does conceptual transfer relate to con- 
ceptual learning, literacy engagement, and 
integrated instruction in grades 3 and 5, when 
background factors are controlled? We ad- 
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Figure 2. Motivated strategy use of CORI and traditional students at third- and fifth-grade. 
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Figure 3. Conceptual learning at levels of CORI and traditional students at third- and fifth-grade. 
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Table 4 












Multiple Regression Analyses for Transfer in the Total Group (Question 6) 








Group Dependent 


Independent 


R 


R 1 


R 1 Cha 


SigCha 


Total I Transfer 


Literacy/Science 


.31 


.09 


.09 


.001 




Prior Knowledge 


.38 


.15 


.05 


.01 




Search 


.45 


.20 


.05 


.01 




Instruction 


.45 


.20 


.00 


ns 


Grade 5 Transfer 


Literacy /Science 


.14 


.02 


.02 


ns 




Prior Knowledge 


.28 


.08 


.06 


.06 




Search 


.50 


.25 


.17 


.0008 




Instruction 


.60 


.35 


.10 


.005 


Grade 3 Transfer 


Literacy /Science 


.45 


.20 


.20 


.001 




Prior Knowledge 


.51 


.26 


.06 


.06 




Search 


.52 


.27 


.01 


ns 




Instruction 


.54 


.29 


.02 


ns 


Total I Transfer 


Literacy/Science 


.32 


.10 


.10 


.001 




Prior Knowledge 


.39 


.15 


.05 


.01 




Write Si Draw 


.53 


.28 


.13 


.0001 




Instruction 


.53 


.28 


.00 


ns 


Grade 5 Transfer 


Literacy/Science 


.12 


.01 


.01 


ns 




Prior Knoweldge 


.24 


.06 


.05 


ns 




Write Si Draw 


.65 


.42 


.36 


.0001 




Instruction 


.68 


.46 


.04 


.05 


Grade 3 Transfer 


Literacy/Science 


.46 


.21 


.21 


.0009 




Prior Knowledge 


.52 


.28 


.07 


.05 




Write Si Draw 


.53 


.28 


.00 


ns 




Instruction 


.59 


.35 


.07 


.03 


dressed this question with 


six multiple re- (p 


< .01) in conceptual transfer. 


Instruction 


gressions using conceptual transfer as the did not contribute significantly to conceptual 

dependent variable. In the first analysis, the transfer after search and background factors 


independent variables were 


the literacy and were controlled. 








science background, prior knowledge, search, 


This part of question 6 was examined sepa- 


and integrated instruction. As Table 4 shows, rately for grades 3 and 5. 


The multiple regres- 


search accounted for 5 % of the unique variance sion for grade 5 using conceptual transfer as 
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the dependent variable with literacy and science 
background, prior knowledge, search, and 
instruction as independent variables produced 
a multiple regression of .60. The search vari- 
able accounted for 17% of the unique variance 
( p < .0008) and instruction accounted for 10% 
of the unique variance (p < .005). In grade 3, 
however, search and instruction did not con- 
tribute significantly to transfer, using the same 
multiple regression procedure used for grade 5 
(see Table 4). 

To address Question 6 further, a hierarchial 
multiple regression was conducted with con- 
ceptual transfer as the dependent variable, 
using literacy and science background, prior 
knowledge, writing and drawing, and instruc- 
tion as the independent variables. For the total 
group, writing and drawing contributed 13% of 
the unique variance in conceptual transfer, 
which was significant ( p < .0001), but instruc- 
tion did not add significantly. When this hierar- 
chial multiple regression was conducted for 
grade 5 and grade 3 separately, a different 
picture appeared. For grade 5, writing and 
drawing contributed 36% of the unique vari- 
ance in conceptual transfer, which was signifi- 
cant (p < .0001); and instruction added 4% of 
the unique variance in conceptual transfer ( p < 
.05). For grade 3, writing and drawing did not 
contribute a significant amount of unique 
variance to conceptual transfer. However, for 
grade 3, instruction added 7% of the unique 
variance in transfer (p < .03) after the factors 
of conceptual learning and student background 
were taken into account (see Table 4). 

These results of the multiple regression 
analyses on conceptual transfer are depicted in 
Figure 4. For the total group, links D and E 



were significant when controlling for links 1 
and 2. For grade 5, links D and E were signifi- 
cant when controlling for 1 and 2. In addition, 
link F (CORI) was significant when controlling 
for 1 and 2, and separately for D and E. For 
grade 3, path E (CORI) was significant when 
controlling for links 1, 2, and D. 

Question 7 

What effect does CORI have on comprehen- 
sion of informational and narrative text? Reading 
comprehension of informational and narrative 
texts were measured with different texts at 
grade-appropriate levels for third- and fifth- 
graders. To examine the effects of CORI on 
comprehension, separate analyses were con- 
ducted using a one-way ANCOVA with liter- 
acy level as the covariate. For grade 5 with 
informational text, the CORI group was not 
significantly higher than the traditional instruc- 
tion group. For grade 5 with narrative text, the 
CORI group was significantly higher than the 
traditional instruction group, F{ 1, 64) = 
15.31, p < .0001. For grade 3 with infor- 
mational text, the CORI group was not signifi- 
cantly different from the traditional group. For 
grade 3 with the narrative text, the traditional 
group was higher than the CORI group, 
F(l,118) = 5.26, p < .02 (see Table 5). 

Question 8 

What instructional principles distinguish 
CORI and traditional instruction? This question 
was addressed in the Method section. The 
dimensions (Scales) on the instructional ques- 
tionnaire were correlated with the dichotomous 
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Figure 4. Promoting conceptual transfer through literacy engagement and integrated instruction. 
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Table 5 

Results on the Performance Assessment 



Variable 


Condition 


N 


Mean 


SD 


Third Graders 










Literature Covariate 


CORI 


39 


2.87 


1.20 




Basal 


39 


3.23 


1.49 


Science Covariate 


CORI 


39 


2.90 


1.55 




Basal 


39 


3.28 


1.32 


Prior Knowledge 


CORI 


42 


2.57 


1.06 




Basal 


36 


2.03 


.77 


Writing 


CORI 


44 


3.23 


1.22 




Basal 


38 


2.39 


1.26 


Drawing 


CORI 


44 


2.95 


1.38 




Basal 


38 


2.53 


1.16 


Writing and Drawing 


CORI 


44 


6.18 


2.19 




Basal 


38 


4.92 


2.05 


Search Relevant Packet 


CORI 


48 


5.46 


1.50 




Basal 


42 


3.93 


1.85 


Search Notes 


CORI 


48 


4.75 


1.00 




Basal 


42 


3.88 


1.27 


Search Reason 


CORI 


48 


4.71 


.92 




Basal 


42 


4.10 


1.27 


Search Total 


CORI 


48 


4.69 


.99 




Basal 


42 


3.64 


1.39 


Informational Text 


CORI 


45 


7.42 


1.54 




Basal 


42 


7.02 


1.87 


Story Comprehension 


CORI 


46 


6.00 


1.62 




Basal 


39 


6.46 


1.41 
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Table 5 (continued) 

Results on the Performance Assessment 


Variable 


Condition 


N 


Mean 


SD 


Fifth Graders 










Literature Covariate 


CORI 


36 


3.31 


1.49 




Basal 


37 


3.35 


.89 


Science Covariate 


CORI 


36 


3.75 


1.27 




Basal 


37 


3.30 


1.18 


Prior Knowledge 


CORI 


36 


2.53 


.84 




Basal 


42 


2.12 


.77 


Writing 


CORI 


35 


4.03 


1.34 




Basal 


40 


2.80 


1.02 


Drawing 


CORI 


35 


3.54 


1.17 




Basal 


40 


2.53 


.99 


Writing and Drawing 


CORI 


35 


7.57 


2.08 




Basal 


40 


5.32 


1.72 


Search Relevant Packet 


CORI 


40 


5.48 


1.52 




Basal 


42 


4.31 


1.44 


Search Notes 


CORI 


40 


5.60 


1.01 




Basal 


42 


4.24 


1.27 


Search Reason 


CORI 


40 


4.65 


1.10 




Basal 


42 


4.38 


.94 


Search Total 


CORI 


40 


5.40 


1.10 




Basal 


42 


4.00 


1.29 


Informational Text 


CORI 


41 


8.71 


1.47 




Basal 


37 


8.65 


1.51 


Story Comprehension 


CORI 


40 


5.90 


1.61 




Basal 


39 


4.67 


1.44 
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variables of instructional type. As Table 1 
shows, correlations were: real-world experi- 
ence (.97), knowledge (.97), direction of 
control (.95), social (.86), strategies taught 
(.61), written expression (.20), and integration 
of language arts and science (.90). This con- 
firmed the distinctions between CORI and 
traditional instruction on six of the seven 
dimensions. 

STUDY II 
Introduction 

Purpose 

The purpose of Study II was to compare the 
effects of CORI with traditional instruction on 
a statewide assessment. The assessment is part 
of the Maryland School Performance Assess- 
ment Program (MSPAP) which is linked to a 
systemic reform effort in education. The 
MSPAP contains measures of achievement in 
reading, writing, language use, science, math, 
and social studies. The goal of this investiga- 
tion was to compare CORI and traditional 
instruction in each of these measures for stu- 
dents at grades 3 and 5 in one elementary 
school. 

Rationale 

The MSPAP is a statewide design for educa- 
tional reform. The design includes: identifica- 
tion of learning outcomes, development of 
performance standards, administration of per- 
formance assessments, implementation of in- 
structional improvements, and publication of 
assessment results. Standards in reading, writing, 



and science were constructed for all students. 
The objectives of the MSPAP and the CORI 
were highly similar. For example, standards in 
reading for both MSPAP and CORI included: 
summarize a story, play, or poem; identify a 
theme of a novel; infer traits and motives of 
characters; restate information from an exposi- 
tory text; identify the organizational structure 
of a selection; evaluate the effectiveness of an 
author’s argument; draw inferences from 
information in illustrations and text; identify 
similarities of tables and graphs; and compare 
different forms of documents . 

In view of the agreement in objectives be- 
tween the MSPAP and CORI, it is reasonable 
that students receiving concept-oriented reading 
instruction should show relatively high achiev- 
ement in the aspects of MSPAP taught within 
CORI. As the CORI objectives are related to 
reading, writing, and science, it was reason- 
able to expect that these tests should reflect the 
effectiveness of CORI. On the other hand, 
CORI did not include instruction in math or 
social studies and therefore, it was not expect- 
ed that these two areas would be influenced by 
participation in CORI. We expected that lan- 
guage use would be influenced favorably by 
CORI because expectations for writing in 
CORI were substantial. 

Although the global objectives of MSPAP 
and CORI are similar, there are several reasons 
to doubt whether the Maryland Performance 
Assessment would show an effect of an instruc- 
tional intervention such as CORI. First, the 
particular reading competencies in the assess- 
ment were not specifically tied to the particular 
competencies in CORI. For example, although 
both teach higher-order reading strategies, 
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CORI emphasizes search for information from 
multiple texts, whereas the Maryland assess- 
ment emphasizes summary of informational 
text. Separate scores in the Maryland assess- 
ment are not obtained for such specific compe- 
tencies as searching and summarizing. In the 
absence of this specificity, it is possible that the 
measure is not sensitive to an intervention that 
targets a few particular higher-order strategies. 
Second, the content of science in the Maryland 
assessment was not related to the content of 
science instruction in CORI. In CORI, life 
sciences were emphasized with some attention 
to earth science and physical science. How- 
ever, in the Maryland assessment physical sci- 
ence, earth science, and life science were 
equally emphasized. Therefore, it is the science 
processes of hypothesizing, data collecting, 
predicting, inferring, and drawing conclusions 
taught in CORI that may be observed in Mary- 
land assessment rather than gains in content 
knowledge. 

Third, student effort for participating in the 
Maryland assessment is not remarkably high. 
Many students fail to see the purpose of these 
activities and do not invest optimal effort in 
them. Fourth, the Maryland assessment re- 
quires substantial levels of self-assessment. 
Students are expected to appraise their own 
strategies for learning, comment on alternative 
methods of performing tasks and reflect on 
their reactions to text. The self-assessments 
were not emphasized in CORI as fully as they 
were in the Maryland assessment. These differ- 
ences between MSPAP and CORI lead to 
realistic doubts about whether the Maryland 
assessment will be sensitive to benefits of 
CORI in the form documented in Study I of 



this investigation. Fifth, approximately 40% of 
the students who received the Maryland assess- 
ment in April 1994 were not present in the 
school for the full 1993-94 academic year. 
Consequently, it is unlikely that the CORI 
program could have influenced their learning 
substantially. 

Method 

Students 

In one elementary school, CORI was imple- 
mented in one grade-5 and one grade-3 class- 
room in the 1993-94 year. Students in these 
classrooms were assigned to the CORI project 
because it was believed that they would benefit 
from special attention to their reading and 
writing development. Both classes included 
approximately 4-6 learning disabled students 
who were maintained in the self-contained 
classroom for reading, writing, and science 
instruction. There were 31 grade-3 and 27 
grade-5 students for whom data were available. 
There were 48 grade-3 and 45 grade-5 students 
from traditional instruction classrooms for 
whom data were available. Students were 
heterogeneously grouped in classes and in- 
cluded approximately 65 % African-American, 
20% Hispanic, 10% Caucasian, and 5% Asian. 
These proportions were roughly representative 
of the district. 

Teachers and Program 

One grade-3 and one grade-5 teacher taught 
CORI in a yearlong integration of science, 
reading, and writing instruction. This was the 
first year of implementation for grade 3 and the 
second year for grade 5. Teachers were sup- 
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ported by a small team of one faculty member 
and one student at the University of Maryland. 
The school was a Chapter One, K-6 school 
with a high proportion of students reading and 
writing in the lowest quartile. 

The instructional framework was the same 
as the framework described in Study I. Instruc- 
tions of the 16-week units contain four phases 
consisting of: (a) observe and personalize; (b) 
search and retrieve; (c) comprehend and inte- 
grate; and (d) communicate to others. In each 
of these phases, teachers supported students in 
generating their learning goals; gaining strate- 
gies to understand the text; reading multiple 
genre including informational texts, narratives, 
folktales, and poetry; and writing extensively 
to communicate their new learning to peers. 
Student portfolios were emphasized as a means 
to exhibit class work for students, teachers, 
parents, and administrators. 

Performance Assessment 

The MSPAP was administered in April 1994 
by the teachers according to the guidelines. 
The students were placed in a room different 
from their homeroom and were given direc- 
tions to perform a timed series of tasks in 
science, social studies, and math during 1 
week. Scores for reading, writing, and lan- 
guage use were derived from the students work 
on content-based tasks. For example, students 
may have performed a hands-on science activ- 
ity collecting and interpreting data. Immedi- 
ately following, they read an informational text 
related to the science concepts inherent in the 
activity and summarized the connections be- 
tween the text and activity. The quality of the 



summary of text was used as a portion of the 
reading score. The qualities of the writing and 
language use in the summary were used to 
contribute to the scores on those scales, 
respectively. 

Results 

To compare achievement of students receiv- 
ing CORI to students receiving traditional in- 
struction, analyses were conducted on each 
measure on the Maryland assessment sepa- 
rately. The means are shown in Table 6 and 
Figure 5. A 2 (types of instruction) X 2 
(grades) ANOVA was performed on reading, 
writing, literacy #1 (sum of reading and writ- 
ing), language use, literacy #2 (sum of reading, 
writing, and language use), science, social 
studies, and math. Each two-way ANOVA was 
followed by a one-way ANOVA on program 
effects at grade 3 and grade 5 separately (see 
Table 6). 

The analysis of literacy showed a signifi- 
cant advantage of CORI compared to tradi- 
tional instruction, F(l, 145) = 4.31 , p < .04. 
The grade-level and interaction effects were not 
significant. The ANOVA for reading and the 
ANOVA for writing did not show any signifi- 
cant effects of the type of instruction or any 
interaction effects. The ANOVA on the lan- 
guage use measure showed a significant benefit 
of CORI compared with traditional instruction, 
F(l, 142) = 3.86, p < .05. There was a 
significant grade effect, F(l, 142) = 8.46, 
p < .0004, but no significant interaction. The 
ANOVA for literacy # 2 , which combined 
reading, writing, and language use, showed a 
significant benefit for CORI compared with 
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Table 6 

Means and Standard Deviations for COR1 and Traditional 
Students on the Maryland School Performance Assessment 



Variable & 
Program 






Students’ 


Grade 






Third 




Fifth 


Third & Fifth 


M 


SD 


M 


SD 


M 


SD 


RD CORI 


503.70 


42.22 


517.43 


50.82 


510.33 


46.67 


Trad 


486.71 


58.72 


504.19 


45.75 


495.18 


53.29 


WR CORI 


508.30 


43.42 


504.15 


51.71 


506.43 


46.96 


Trad 


491.27 


41.98 


494.91 


45.34 


492.98 


43.40 


LU CORI 


507.72 


43.86 


519.81 


38.53 


513.55 


41.45 


Trad 


484.09 


40.68 


512.89 


57.03 


498.17 


51.17 


MS CORI 


518.69 


28.93 


510.96 


25.64 


514.96 


28.93 


Trad 


499.41 


45.55 


512.34 


45.80 


499.41 


45.55 


SS CORI 


508.31 


35.06 


511.39 


32.70 


509.82 


33.65 


Trad 


490.34 


43.60 


502.24 


45.00 


495.98 


44.44 


SC CORI 


515.29 


25.42 


521.81 


30.60 


518.33 


27.89 


Trad 


492.71 


43.26 


511.04 


45.58 


501.58 


45.11 


Litl CORI 


1014.83 


65.40 


1024.15 


82.40 


1019.32 


73.55 


Trad 


984.62 


83.03 


999.63 


77.83 


999.04 


80.42 


Lit2 CORI 


1522.55 


95.90 


1543.96 


106.80 


1532.88 


100.95 


Trad 


1465.85 


111.51 


1514.11 


127.37 


1489.44 


121.30 



Note. RD = Reading; WR = Writing; LU = Language Use; MS = Mathematics; SS = Social Studies; 
SC = Science; Litl = Literacy (RD=WR); Lit2 = RD+WR+LU. 



traditional instruction, F(l, 142) = 5.17 ,p < 
.02. On literacy #2, a significant effect for 
grade appeared, F(l, 142) = 4.12,/? < .04, 
but the interaction was not significant. 

On the science scores, the A NOVA showed 
a significant advantage for CORI compared 



with traditional instruction, F(l, 147) = 6.80, 
p < .01. A significant grade effect appeared, 
F(l, 147) = 4.73,/? < .03, but the interaction 
of instruction and grade was not significant. A 
significant advantage for CORI also appeared 
on the social studies measure, F(l, 148) = 4.03, 
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Figure 5. Comparison of CORI and traditional students on reading, writing, language use, math, 
social studies, and science in the Maryland Performance Assessment. 
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Table 7 

Comparison of CORI and Traditional Instruction on the Maryland Performance Assessment 



Measures 



Total (Grades 3 & 5) 



Reading 

Writing 

Literacy ft\ (R+W) 

Language Use 

Literacy HI (R+W+LU) 

Science 

Social Studies 

Math 



ns 

ns 

* 

* 

* 

* 

* 

ns 



Note, ns = Not significant; * = Significant instructional effect at p <.05. 



p < .05, but there were no grade or interaction 
effects. On the math measure, there were no 
significant effects for instruction, or grade, on 
the interaction of instruction and grade. 

The interaction of instruction and grade was 
not statistically significant in any of the analy- 
ses, which signifies that the advantage for 
CORI in achievement was not significantly 
different for grade 3 and grade 5. Therefore, 
separate grade-specific analyses are not war- 
ranted according to strict criteria. 

These comparisons of CORI with traditional 
instruction were consistent with our expecta- 
tions (see Table 7). Students who received 
CORI were higher in literacy than students 
receiving traditional instruction. The literacy 
measure was a combination of the reading and 
writing scores from the Maryland assessment. 



Combining them is consistent with the philoso- 
phy of CORI and with the goals of the systemic 
change that include the integration of reading 
and writing for authentic school learning pur- 
poses. 

Students receiving CORI showed higher 
achievement in science than students receiving 
traditional instruction, this was expected be- 
cause CORI is an integration of language arts 
and science for elementary school learners. In 
addition to science, CORI students were higher 
than traditional students in language use. This 
was a measure of the skills and craft of writ- 
ing, including sentence structure, capitaliza- 
tion, and word use. The advantage for CORI 
showed that CORI improved skills as well as 
higher-order strategies in writing. 
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Math achievement of CORI students was not 
higher than math achievement of traditional 
students. This finding provides valuable con- 
verging evidence, because math was not in- 
cluded in the objectives of instruction. Since 
CORI students were higher than traditional 
students in literacy and science, but not higher 
in math, the program effects are specific. This 
finding suggests that the advantage of CORI 
was not a consequence of higher academic 
ability, more motivation for schooling, English 
language competencies, or the characteristics of 
teachers that were uniquely associated with the 
CORI classrooms. From this pattern, we infer 
that it was the CORI program rather than 
external factors that may be the sources of 
enhanced achievements. 

It should be noted that CORI students sur- 
passed traditional students on the social studies 
measure in the MSPAP. Although this finding 
was unexpected, it is not mysterious. Reading 
and writing were necessary for high scores on 
the social studies measures. CORI students 
were taught to handle large texts with multiple 
interpretation and to compose extended, thought- 
ful reactions. These competencies were likely 
to be valuable in the social studies tasks of the 
performance assessment. 

CORI is not a finished product. The nature 
of the teaching, required materials, profes- 
sional development activities, administrative 
supports, and other curricula requirements are 
yet to be fully determined. Despite the needs 
for continued improvement, it is plausible to 
suggest that CORI is likely to enhance achieve- 
ment in grades 3-6 on broad, integrated perfor- 
mance assessment measures of achievement. 



DISCUSSION 

This paper was addressed to the broad ques- 
tion of “Does CORI increase motivation, 
strategies, and conceptual learning?” A variety 
of researchers, administrators, and teachers 
have remarked that CORI appears to be an 
attractive framework for integrating teaching 
and motivating higher-order learning in literacy 
and science. In the same breath, these observ- 
ers often ask whether this CORI increases 
achievement in comparison to more traditional 
teaching. To explore this issue, we conducted 
two quasi-experiments comparing students who 
received a year of CORI with students who 
received a year of traditional teaching in liter- 
acy and science. In both Study I and II, an 
extended posttest in the form of a performance 
assessment, administered in the spring of the 
academic year, was used to compare achieve- 
ment of the experimental and comparison 
students. 

In Study I, we introduced statistical controls 
of literacy level, science background, and topic 
knowledge on the content of the performance 
assessment to increase the comparability of the 
populations. Comparisons showed advantages 
for CORI. As Figures 2 and 3 show, third- 
grade students in the CORI program had higher 
performance on measures of engagement and 
conceptual learning than third-grade students in 
the traditional classroom program. Likewise, 
fifth-grade CORI students had higher levels of 
literacy engagement and conceptual learning 
than fifth-grade traditional students. 

We observed an additional, unexpected 
finding. Students in third grade who received 
CORI showed higher achievement than stu- 
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dents in fifth grade who participated in a tradi- 
tional teaching program. This advantage for 
CORI appeared for literacy engagement, which 
we defined as the motivated use of cognitive 
strategies for learning science concepts. In 
Study I, the search score on the performance 
assessment was used as the indicator of literacy 
engagement. High search scores indicated that 
students located appropriate text, extracted 
relevant information from these texts, and 
persisted in elaborating their understanding of 
the conceptual topic. 

The relatively high performance of CORI 
third-grade in comparison to traditional fifth- 
grade students was also observed in conceptual 
learning. Students were more able to marshal 
relevant facts, provide cogent explanations, and 
link concepts theoretically if they participated 
in CORI. It should be noted that these infer- 
ences are possible only if the students at grades 
3 and 5 are placed on the same scale with the 
same performance assessment. In this study, 
the performance assessment with texts, tasks, 
writing expectations, and prompts provided a 
learning and testing environment that was 
equitable for students in both programs at both 
grade levels. Students in traditional teaching, 
as well as CORI, were being taught similar 
science contents and science processes. 

An alternative framework for showing the 
findings of Study I is presented in Figure 1. 
Combining the results from all of the students, 
it shows that conceptual learning was substan- 
tially increased by motivated use of cognitive 
strategies. Students who became more fully 
engaged in learning science from studying 
multiple texts, attained higher levels of science 
knowledge than students who were less en- 



gaged. In addition, Figure 1 depicts that stu- 
dents who received CORI increased in literacy 
engagement more than students who did not 
receive CORI. The principles of CORI appear 
to enhance cognitive competence and motiva- 
tional dispositions for higher-order learning. 
These capabilities were transferred from the 
curriculum of the school year to the perfor- 
mance assessment context. These findings, it 
should be noted, were maintained when student 
background variables of literacy levels, science 
knowledge, and topic knowledge were held 
constant statistically. 

An intriguing finding was that CORI en- 
hanced conceptual learning above and beyond 
its influence on literacy engagement. CORI 
students were more successful than traditional 
students in integrating their thinking, writing 
cogently, and providing full, theoretical expla- 
nations of the topic. Not only did they search 
more efficiently and display intrinsic motiva- 
tions for learning during the search activity, 
they were able to use the information gleaned 
in their search to write more integratively than 
traditional students. Another way of stating this 
finding is to say that although a substantial 
proportion of the effects of CORI on concep- 
tual learning were mediated through literacy 
engagement, some benefits of CORI on con- 
ceptual learning were direct. 

In Study II, we found that CORI students 
were higher than traditional students on the 
literacy and science sections of the Maryland 
School Performance Assessment. This finding 
adds credence to the results of Study I. Be- 
cause CORI students did not achieve higher 
scores in areas that were not taught (e.g., 
math), the achievement differences seem attrib- 
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utable to effects of the instructional program 
rather than student background or teacher 
characteristics. Although Study II did not 
statistically control background variables as 
Study I did, the convergence of results in the 
two investigations tends to confirm the effec- 
tiveness of the instructional program for en- 
hancing literacy engagement and conceptual 
learning. 

How does CORI compare with other forms 
of integrated teaching designed to increase 
conceptual learning? Integrated teaching is 
increasingly popular in schools. A large pro- 
portion of language arts teachers are connecting 
the reading, writing, speaking, listening, and 
communications in language arts instruction. 
Many teachers are also integrating disciplines 
with each other. For example, language arts 
and history are often linked around topical 
themes; science and language arts are taught in 
conjunction by emphasizing reading and writ- 
ing in expository texts and trade books along 
with “hands-on” science activities (Saul et al., 
1993). Full integrations of science, history, 
English/language arts, math, and geography 
are occasionally attempted (Beane, 1995). Al- 
though a variety of instructional integrations 
are being implemented (Stephenson & Carr, 
1993), relatively few attempts have been made 
to describe them thoroughly and even fewer 
attempts have been made to examine the im- 
pacts of integrated teaching on student learning. 

At least three distinct forms of integrated 
teaching can be compared with CORI. In the 
project-based approach, teachers help students 
to form a thematic learning goal (Blumenfeld, 
Soloway, & Marx, 1991). Students engage in 



high amounts of self-directed activity in small 
groups and individually to attain the goal. A 
prominent aim within the project-based ap- 
proach is to construct an artifact that embodies 
the knowledge gained in the project. The 
artifact may be a diorama, a model, or a depic- 
tion of relationships in the form of a poster or 
video. CORI is contrasted from project-based 
teaching because the basic aim in CORI is 
conceptual. In CORI, the artifact in the form of 
a poster or video is not considered a major 
objective, but is considered one of several 
forms of documenting progress toward the 
more conceptual aim of the teaching unit. 

Integrated teaching described by Brown 
(1992) is directed to in-depth learning about 
broad concepts of environmental science (e.g., 
ecological interdependencies). A major goal of 
the Brown (1992) program is transfer of con- 
ceptual knowledge from the particular topic of 
instruction to new allied topics. Teachers 
emphasize collaborative teams with distributed 
expertise across different members of the team. 
Although CORI also contains an emphasis on 
conceptual learning through team-based in- 
quiry, CORI places a higher emphasis on 
direct strategy instruction, of multiple genre 
including novels and folktales, and literacy as 
well as science learning. 

Duffy (1993) has designed instruction in 
strategies for learning substantive content. In- 
struction in comprehension, integration, and 
composition was relatively individualized, and 
cognitive-strategy-centered. In comparison, 
CORI places relatively more emphasis on con- 
ceptual understanding, collaborative interac- 
tion, self-expression, and interdisciplinary 
structure of content. 
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In conclusion, CORI draws upon the proj- 
ect-based emphasis of Blumenfeld et al. (1991) 
to underscore the importance of self-directed 
learning toward intrinsically interesting goals. 
CORI relates closely to Brown’s (1992) empha- 
sis on in-depth conceptual learning as a basis 
for transfer and problem solving. CORI incor- 
porates the direct coaching of strategies for 
understanding text and learning science con- 
cepts exemplified by Duffy (1993) and Collins- 
Block (1992). In addition, CORI places a 
prominence on real-world experience. Hands- 
on activities are linked to literacy and con- 
ceptual goals. Self-expression of conceptual 
knowledge through informational writing, 
narrative compositions, video portrayals, 
posters, and dioramas are explicitly intertwined 
into the learning experience of students. Fi- 
nally, the hallmark of CORI is coherence of 
experience. Students link hands-on experience 
with text-based knowledge, individual endeavor 
with collective effort, science explanations with 
literate tools for inquiry, and school-relevant 
curricular objectives with personally significant 
interests. Finding ways to enhance the coher- 
ence of the classroom is one of our main, 
continuing goals for instructional design. 
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Appendix 

Rubrics for Coding the Performance Assessment 

Prior Knowledge 

This rubric was the same as the writing rubric. 

Writing 



Level 1 

At this level the students presented: (1) no information; or (2) scientifically inaccurate 
information; or (3) one appropriately identified feature or specie for each biome; or (4) two 
features or species for only one of the biomes. 

Level 2 

Students presented a combination of 2-4 general or specific features of the biomes or species 
for at least one biome. However, no relationships among them were demonstrated. At this level, 
students may also present scientifically inaccurate information. 

Level 3 

At this level, the student may have presented accurate and relevant information for a minimum 
of four specific features of one biome. An alternative was that students could present any 
relationships or connections between the species and the features of the biome, but the relationships 
were implicit through comparison or vague. No explanations for relationship of features are 
presented. 

Level 4 

At this level, a student presented all of the necessary elements for a level 3, but the relationships 
or connections between species and the features of the biome were presented explicitly. However, 
these relations were stated for only one biome and the other remains implicit or vague. Students 
may also present their answer in the form of a clear relationship in one biome that was not trans 
ferrable to the other. Explanations were explicitly included and were distinguishable from 
particular facts or statements of biome and species characteristics. 
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Level 5 

At this level, students wrote an explicit relationship between species and features for both 
biomes. The comparison was often parallel, with one type of animal in one biome comparable to 
a type of animal in the other biome showing asymmetry. However, answers at this level lack 
connections between the relationships between the species and features. An alternative means for 
obtaining a level 5 was to present a systemic (level 6) response for only one biome with very 
limited information on a second biome. 

Level 6 

At this level, students clearly explicated the principles that link the features of biomes in terms 
of “systems.” For example, life cycles or adaptations of certain species and the relationships of 
these species and features of their respective biomes were stated. The systems are governed by 
principles centered around the availability of water, temperature, formations of the biome, sur- 
vival, or food chains. 



Drawing 

This rubric was the same as the writing rubric. 

Search 

The rubric for search consisted of six levels. Several aspects of the log were incorporated into 
the final reading including the number of relevant packets chosen by the students, the quality of 
the notes evidenced in the log, and the characteristics of the reasons given for choosing the 
packets. Students’ motivational characteristics such as elaboration, persistence, and apparent 
interest in task were recognized. 

Level 1 was assigned to student work in which responses were nonexistent, incoherent, or 
completely irrelevant to the question. Reasons for choosing the packet were not presented. 

Level 2 was assigned to responses in which students reported 1-3 relevant packets. The notes 
however, showed vague information related to one plant or animal. Reasons for choosing the notes 
were excluded or were self-referenced (such as, “I liked it”). 

Level 3 was assigned to responses in which 3-5 relevant packets were selected and several 
irrelevant packets (1-3) may have been included. Notes appeared in the form of “captions” or titles 
and headings. Usually 3-6 features or species were listed and a number of these features may be 
scientifically trivial. One packet usually showed the notes that reflected a written reason for 
choosing them. However, the links between notes and packets were otherwise missing, incoherent, 
or repetitive. 
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Level 4 consisted of 2-5 relevant packets. From these a substantial number of appropriate 
structures (3-15) for each biome was identified including plants, animals, weather, and soil of each 
biome. 

Level 5 was assigned to responses in which 4-7 relevant packets were selected. The key 
structures of both biomes of pond and desert were included and a few simple relations between 
particular animals or between specific plants and biome features were expressed. 

Level 6 consisted of 4-7 relevant packets with an explicit reason for them. The notes illustrated 
the principle of adaptation for both plants and animals. At this level, life cycles of plants and 
animals are shown be related to the features of biomes. The sequence of packets selected during 
the search showed cumulative development of biome and adaptive information about plants and 
animals. The notes may have possessed the quality of “argument” if information is combined with 
reasoning to present a cogent viewpoint. The notes consistently contain information that fulfills the 
goal stated in the reason. 

The reasons for choosing packets were coded separately and combined with the initial levels 
assigned to packet selection and quality of the notes. 

Reasons at Level 1 were not presented or were not distinct from notes. 

Reasons at Level 3 were distinct from notes but were not topical or conceptual (e.g., “it was 
interesting”). Reasons were not associated or connected to the content of the notes. 

Reasons at Level 5 consisted of 4-7 reasons stated distinctly from the notes. However, the 
reasons were too general or too specific although they were related to the content of the notes for 
a majority of the packets. 

At Level 6, 5-9 reasons were presented. These reasons consisted of a goal for reading that was 
at least as specific as the title for a major heading of the text, but was not a verbatim repetition. 
Reasons were presented in a definable sequence and related to the notes. 

These levels of supplying reasons for choosing information were related to the initial coding. 
Having identified the rubric level based on selections and notes the coder then adjusted the level 
based on the coding for reasons according to the following table: 



Selection and Notes 


Reasons 


Final Code 


7,6 


5 or 7 


no change 


7,6 


3 or 1 


reduce 1 


5,4 


7 or 5 


no change 


5,4 


3 


reduce 1 


5,4 


1 


reduce 2 


3,2 


3 or 5 


no change 


3,2 


1 


reduce 1 


1 


1,3 or 5 


no change 
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Transfer 

This rubric was the same as the writing rubric. 

Information Text Comprehension 

Each question in this section was scored on a three-point scale (1-3). The scores range from 
4-12. The general standards for each answer consisted of the following: 

1. Inaccurate or an incoherent answer; the answer did not demonstrate an understanding of the 

question or the text. 

2. Accurate response demonstrating an understanding of either of the text or the question. 

3. Accurate and elaborated response demonstrating a complete understanding of both the text and 
the question. 

These expectations were applied to both grade-3 and grade-5 responses. For grade 3, Level 1 
responses were consistently incoherent or irrelevant to the question. For grade 5, Level 1 
responses were incoherent, inaccurate, or extremely vague and general with one or two remotely 
relevant pieces of information. 

Level 2 responses for grade 3 included a few important bits of information from the text and 
one relevant point that addressed the question. For grade 5, Level 2 responses included a portion 
but not all of the text-based information needed in the question. Relations between information and 
explanations were absent or weakly stated. 

Level 3 responses for third-grade included essential concepts, accurate information, and a 
completeness of description. For grade 5, Level 3 responses contained full descriptions with 
relations among concepts clearly defined. Explanations were provided if requested in the question. 
The response contained critical details and elaborated statements of their relationships to each 
other. 

Narrative Interpretation 

The three questions in the narrative assessment task were each coded on three levels. This 
provided a score of 3-9 on the narrative task for each student. 

At Level 1, the students provided an irrelevant answer, an incoherent brief statement, no 
answer, or a self- reference but not a text-reference response. 

Level 2 responses for grade-3 and grade-5 students included appropriate information that was 
vague, limited in scope, and/or marginally appropriate to the question. Level 3 responses for 
grade-3 and grade-5 students included elaborate descriptions that were question-relevant and text- 
based. These students supported a general point with specific examples from the text and replied 
to all parts of the question. 
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